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ABSTRACT 

The purpose of this study was to establish a model 
for the evaluation of the quality of vocational education programs in 
Tennessee, A definition of educational quality was developed from a 
literature search and from a suirvey of teacher-educators and 
super»/isory perscmnel. Principals of 54 secondary schools identified 
as having vocational education programs with quality characteristics 
were sent forms to be completed by their guidance counselors and 
teachers* Each respondent indicated the elements that he considered 
important to quality* From these data 20 quality components were 
established and 60 evaluative items (3 to measure each component) 
were developed to evaluate vocational programs. These items were then 
made part of an emalysis of the degree of correlation between 
specific programs and the consensus ^'quality** program. Research teams 
collected data during site visits to 18 representative schools 
selected from the initial 54. Summaries of the responses and copies 
of the forms used are appended, along with a detailed analysis of all 
of the data. It was recommended that the proposed model be used in 
evaluating all vocational education programs in Tennessee and that 
graduates of these programs be followed up to keep the programs 
current. (MF) 
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SECTION I 
INTRODUCTION 



With the trend in public education to intensify, enlarge, and improve 
vocational-technical programs, the impact of Federal legislation and sup- 
port, and the current national emphasis on Career Education at all grade 
levels, evaluation of vocational-technical programs has taken on added sig- 
nificance. 

In the :971 Annual Evaluation Report of the Tennessee State Advisory 
Council, the Council recommended the establishment of a more comprehensive 
evaluation plan to evaluate programs on the basis of quality and process- 
along with product and that the plan be implemented at the earliest feasible 
date. 

The initial review of literature concerning the measurement of quality 
in vocational-technical education revealed that relatively little had been 
done in the area and that few ideis concerning quality programs had been 
researched. Evaluation has failed to develop as fast as some other phases 
of education. A lack of well-designed instruments and appropriate procedures 
for gathering data is evident. 

Evaluation methodology, which has been commonly ujed in vocational- 
technical education now fails, in many instances, to provide evidence 
needed by decision-making groups and must be replaced by more effective 
methods. In this day of "accountability," evaluation is necessary to plan 
and/or redirect vocational- technical programs. To keep "up to date," 
vocational-technical personnel must assess the effectiveness of their total 
programs in light of present needs. The evaluation should be thorough, 
systematic, and continuous. Evaluation should be viewed as a way to improve 
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and modify programs, not just condemn programs. 

The challenges and opportunities for all persons involved in 
vocational-technical education have never been greater. 

DEVELOPMENT OF THE STUDY 

In the initial stages of this study every effort was made to involve 
as many of the persons connected with vocational-technical education in the 
state of Tennessee as possible. The research team realized that the 
implementation of the results would depend on the cooperation of these 
potential users—State Department personnel, administrators, teachers, 
and teacher-educators. These persons also possess a wealth of information 
and experience which would be valuable in developing a realistic evaluation 
model for use in Tennessee. 

After a consensus definition of "quality" and the most important com- 
ponents of a quality vocational -technical program were established, the 
research team developed an evaluative instrument based on 20 components. 
One of the aims of the study was to develop an evaluation model that couJd 
be easily understood and used by teachers, administrators, supervisory 
personnel, and students. It is logical to assume that a major responsibility 
for collecting data to use in evaluation of vocational-techiiical programs 
is placed on the teachers in the local schools. Therefore, the amount of 
time required for use of the instrument was a factor. 

Within the limitations of this stady, the material was constructed to 
apply to all vocational-technical programs. One program area clarification 
had to be made. When the field of Home Economics is considered, it refers 
only to Vocational Home Economics. To simplify the procedures for this 
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study^ it was decided to include only secondary vocational-technical 
programs . 

The use of the program evaluation model developed in this study, in 
conjunction with existing evaluative measures, should increase the effective 
ness of the evaluation of vocational-technical programs, as well as 
provide direction for improving the quality of each program. 

PURPOSE OF THE STUDY 

The basic purpose of this study was to establish a model for the 
evaluation of the quality of vocational-technical education programs in 
the state of Tennessee. The two objectives of the study were: 

1) to determine an acceptable definition of "quality" in vocational- 
technical education; and 

2) to test the worlcability of this definition in selected secondary 
vocational -technical programs across the State. 

PROCEDURES 

Initial review of related literature indicated a scarcity of meaning- 
ful definitions of "quality" vocational-technical educational programs and 
evaluation instruments for use by local, regional, and state personnel. 

This study was conducted in two phases as defined by the objectives. 

From the last five years' writings by leaders in vocational-technical 
education, a summary of relevant points dealing with quality in education 
was made. After several revisions, a draft definition of a "quality" 
secondary vocational-technical education program was developed from the 
ideas of these "experts." 



Concurrent with the development of this definition, the research team 
summarized from the available literature 20 of the most important components 
of a quality vocational-technical secondary program. A copy of the draft 
definition and components is included as Appendix A. 

Knowing this was a library research effort at this stage and lacked 
validity in terms of field inputs, the researcher circulated the draft 
definition and components to teacher educators in vocational-technical areas 
at the University of Tennessee, Knoxville. This review represented the fi^st 
level of. input from field personnel. It was hypothesized that these persons 
could provide pertinent input since they work with students, teachers, super- 
visory personnel, and other agencies throughout the State. During this 
procedure the material was also presented to several regional vocational- 
technical supervisory personnel for reactions and review and a check for 
compatibility with State goals. 

The research team then synthesized the comments from these respondents 
to produce the tentative definition of "quality" vocational -technical 
programs and components for quality vocational-technical programs. (See 
Appendix B.) 

Visits were made to the three State regional vocational-technical 
offices to meet with supervisory personnel. The researcher discussed the 
nature of the study and their expected involvement. The researcher had 
previously discussed the project with State Department officials and 
received permission to involve each region in the study. 

At this point, the regional supervisory personnel played a key role. 
The research team made the decision, based on available literature review 
and research efforts, that the identified components of a quality vocational- 



technical program would best be verified in secondary schools with above- 
average programs. Since there was no specific measure in the state of 
Tennessee for quality vocational-technical programs, each regional super- 
visor was asked to identify four programs in his program area (Agricultural 
Education, Distributive Education, Hcctlth Occupations, Home Economics, 
Office Occupations, or Trade and Industrial) which he considered to have 
"quality" characteristics. Where thert was more than one regional super- 
visor in an area, the group was asked .to identify those programs which were 
most representative and contained the elonents which they viewed as being 
vital to a quality program. From the four programs submitted, the researcher 
drew at random thr^e programs from each area. This sample provided 18 
programs in each region or 9 programs in each area. The map in Appendix C 
shows the location of the identified programs. 

The next step in Phase I of the study was to write the principals of 
the 54 schools explaining the purpose of the study and how schools were 
identified for participation in the study. They were asked to cooperate 

c 

through participation. Each principal was sent a packet of materials and 
requested to distribute enclosed copies of the instmment (tentative consensus 
definition and 20 components) to the guidance counselor (s) associated with the 
vocational- technical program and to the teacher (s) of the vocational -technical 
area identified in his school. Each respondent was asked to react to the 
tentative consensus definition and to indicate the importance he places on each 
component by checking one of the following: "Essential," "Highly Desirable," 
"Desirable," "Of Some Importance," "Of No Importance," and "No Opinion." The 
completed forms were to be mailed individually to the research team at 
Knoxville. Follow-up was undertaken when the original deadline was not met. 



A copy of the letter which was sent to the principals is included as 
Appendix D. 

Copies of the tentative definition of "quality" vocational-technical 
programs and lists of components were sent to all regional supervisors of 
vocational-technical education in the State. 

The researcher attended the spring meeting of the State Advisory 
Council on Vocational -Technical Education and explained the research project, 
distributed copies of the tentative definition and components, and requested 
the assistance of the Council members. 

Each of these persons was also asked to react to the content of the 
tentative definition and comment on its acceptability and workability, 
rate the components, and add any components he felt should be included. 

Several persons responded to the request and their ideas were considered 
in the revision of the definition. None of this large group of respondents 
suggested the addition of a component or a major change in any component 
which the research team felt altered the tentative components list. 
However, the research team decided that Component 15 would be better stated 
in a more generalized form. 

The varied experiences of these teachers, principals, guidance 
counselors, teacher-educators, state and regional supervisors, and State 
Advisory Council members provided valuable input for the formulation of 
the consensus definition and components proposed for use in Tennessee. 

The revised consensus definition of a "quality" vocational-technical 
program follows: 



A "quality" vocational-technical secondary program 
will provide current occupational information, enroll 
an interested, qualified student (regardless of his 
mental or physical potential), assess his needs and 
abilities, and prepare him for successful entry-level 
employment (without additional training in the same 
skills)* and place him in the occupational "cluster" 
for which he, his parents, the vocational teacher, 
guidance counselors, and other teachers have jointly 
agreed he should train. "Quality" programs will 
always strive to satisfy basic needs and to develop job- 
getting, job-keeping, and job-advancing skills. 

*The goal might be defined to provide training which 
would prepare him for entry into post-secondary and/or 
college training in his field. 

The components for "quality" vocational-technical programs are: 

1) Complement, not conflict with the objectives of the total 
educational program. 

2) Have verbal, visible support of administration, guidance 
counselors, and other teachers (as much as academic programs). 

3) Provide broad-based testing program and vocational guidance 
services so that each student can select a proper vocational 
program. 

4) Arrange parent-student-teacher conference when student is 
enrolled and throughout his program, if feasible. 

5) Provide flexibility for student to move within program if 
need and/or ability indicates a move is advisable. 

6) Provide physical facilities, equipment and instructional 
materials appropriate to occupational "cluster" for which 
student is being trained. 

7) Use only teachers who have completed state certification re- 
quirements for area in which they teach, who have recent 
related occupational experience, who are proficient in skills 
they teach, and who engage in professional growth activities. 

8) Provide cooperative or directed occupational experience and/or 
simulated occupational experience. 

9) Actively use "representative"advisory committees in occupa- 
tional needs assessment, program planning, and evaluation. 

10) Conduct periodic follow-up surveys of former students and keep 
up-to-date records for use in program planning and evaluation. 



11) Make periodic surveys of employment opportunities and use 
findings to keep curriculum relevant* 

12) Have plan of continuing and systematic internal evaluation 
for improvement and development of program, 

13) Endorse and implement active leadership development 
program through related vocational youth organizations. 

14) Develop proper attitudes toward work, good employer -employee 
relationships, and efficient work habits and create the 
desire to continue personal and occupational development, 

15) Stress each student's total development within his mental 
and physical abilities, 

16) Provide for satisfactory completion of defined program for 
at least 90 percent of students who do not transfer or with- 
draw for health reasons, 

17) Place students at the completion of their programs in the 
occupational "cluster" for which they are trained. 

18) Make an organized school placement service available to 
graduates . 

19) Keep public informed- -have "participatory" support of community. 

20) Use any available means to acquaint students at all levels 
with vocational-technical career opportunities. 

The instrument appears in Appendix E, 

The research team compiled the responses from the mailing to the 54 
selected programs to establish criteria based on local programs. 

Table 1 presents a summary of the responses for each program area and 
for each region. These data show a return from East Tennessee schools of 
98 percent; from Middle Tennessee schools, 93 percent; and from West Tennessee 
schools, 93 percent. The overall return was 94 percent. 

These data were used to establish the identification of each component 
as "Essential," "Highly Desirable," "Desirable," "Of Some Importance," or 
"Of No Importance," An identification was established for total responses 
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RESPONSES TO "COMPONENTS" INSTRUMENT 
BY PROFESSIONAL GROUP BY STATE REGION 



Program/Region 

• 


Principal 


Teacher 


Guidance 
Counselor 


■ 

Agriculture 








East 


3 


3 


3 


Middle 


i 


3 


3 


iVcS u 






9 


Distributive Education 








East 


3 


3 


3 


Middle 


2 


2 


2 






X 
o 


0 


Health Occupations 








East 


3 


3 


3 


Middle 


3 


3 


3 


West 








Home Economics 








East 


3 


3 


3 


Middle 


3 


3 


3 


West 


3 


3 


3 


Office Occupations 








East 


2 


3 


3 


Middle 


2 


3 


3 


West 


3 


2 


3 


Trade and Industrial 








East 


3 


3 


3 


Middle 


3 


3 


3 


West 


3 


3 


3 
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and for responses from each program area* A detailed analysis of these 
data will appear later. 

Identifying and ranking components of a quality vocational-technical 
program has limited utility without devising specific measurements. Realizing 
what a monumental undertaking this could be, the research team decided to 
seek a set of evaluative items which would be representative of all vocational* 
technical areas. 

Concurrent with the securing of responses concerning the definition and 
20 components from principals, teachers, and guidance counselors across the 
state, 60 items (three to measure each component) were developed to evaluate 
vocational -technical programs on the basis of the 20 components. The items 
were written, reviewed and revised by vocational-technical teacher-educators, 
curriculum specialists, vocational-technical doctoral students, and research 
team members. 

It was hoped that these items could be used by principals, teachers, 
regional supervisors, and/or State Department personnel as part of an analysis 
of the degree of correlation between a specific program and a consensus 
"quality" program. The items were to be checked "Excellent; very well done," 
"Satisfactory; adequate," "Some Improvements needed; no crucial weakness (es)," 
"Major improvements needed," "No effort made," or "No opportunity to observe; 
not applicable" on a High (5) to Low (1) scale with (0) for the last category. 

This instrument was administered^to] teachers and students in field 
tests in the East Tennessee area to determine the instrument's content 
validity. Revisions were made where appropriate. A copy of the final 
"Self Evaluation for Vocational -Technical Programs," consisting of 60 items, 
is included as Appendix F. 



From the initial 54 programs, one program in each area from each 
region was selected to participate in Phase II. Programs selected included 
only those from whom responses to the initial mailings were received from 
all three professional groups. IVhere possible, a geographic and rural-urban 
breakdown was maintained. It was also decided that not more than one 
program per school would be included in the final sample. In this manner, 
18 schools were selected to participate in the second phase of this study. 
These schools are listed in Appendix G. 

The research team for Phase II included vocational- technical students 
in doctoral programs at the University of Tennessee, Knoxville. The team 
members made a scheduled visit to each school; took copies of the 60**item 
checklist to the principal, guidance counselor (s) and teacher (s); and 
administered the instrument to senior-level students in the field under 
study. The completed forms were returned individually to the researcher if 
not brought back by the team member. An integral part of this site visit 
was a report from the research team members on their perceptions of the 
program and other observed related factors. 

A detailed analysis of data from the "Self Evaluation for Vocational- 
Technical Programs" by professionals and students will be presented in a 
later section of this report. These data will be combined with data from 
Phase I in an overall analysis. 

Additional general information was gathered by the team member from 
the principal, teachers, and counselors in visitation throughout the school. 
A copy of the questionnaire used is found as Appendix K. A detailed 
summary of these data is presented later. The researcher felt that the 
insights provided by these vocational-technical graduate students 



who combined recent field experience with current campus study would 
provide great depth in determining a workable evaluation model. 

A final item of information secured from the visits was a list 
of advisory committee members, employers, or "lay*' persons who were familiar 
with the program under consideration. The research team felt that these 
persons could contribute valuable input to expanded evaluation activities. 



SECTION II 
m'lm OF RELAltD IJTEPu\TUPE 



An early 1972 survey of the last five years' literature concerning 
vocational-technical education produced many references to quality in 
vocational-technical programs, the need to strive for qufility. the need to 
evaliiate to determine qt-alsty. etc., bur. no one dcfinr'^ a "quality" 
vocational-technical program. 

In reviewing the many articles, speeches, project reports, workshop 
suranaries, and program area corricular and evaluative mnrrrials concerning 
vocational education evaluation, these general points appear many tines: 

There is a need to improve vocational ec?ncr.tion evriuation. 

Vocational-technical programs must be evaluated to j^nrovs proerams 
and to insure the highest possible quality for programs. 

To meet the needs of youth and industry vocational education curriculum, 
personnel, instruction, and mothers mn.st be contimionsly evaluated. 

Evaluation must be pJanre^l, thorough, and continuous. 

Evaluation results provide much-needed inmt for program planning 
decision making, answering '^accountabilitr'- questions, and keening 
goals and objectives current. " 

The Vocational Education Act of J963 and the Vocational Education 
Amendments of 1C68 have had tremendous impact on tho dovelopment of 
evaluation systems at federal, state and local levels. 

There is a shortage of manpower, time, and competency to design 
evaluative instruments, and collect and analyze data. 

Post evaluations havo focused on the educational proces.*;— curriculum 
organization, staff activities and qualifications, physical facilities— 
and tended to ignore program Kiputs and program cutcomos. 

To the researcher, however, those models or systems for evaluation that 

have been developed seem complex and cumbersome with broad objectives and 

would require considerable work and knowledge for implementation. Many 



evaluative schemes place emphasis on '*head counts/' square footage, 

ntionber of machines, etc., at the expense of program outcomes. 

The Vocational Education Amendments of 1968, 

"authorizes Federal grants to States to assist th^ro to main- 
tain, extend, and improve existing programs of vocational 
education, to develop new programs of vocational education, 
and to provide part-time employirient for youths who need the 
earnings from such cmployipcnt to continue rJieir vocational 
training on a full-time basis, so that persons of all ages 
in all communities of the State--those in high school, those 
who have completed or discontinued their formal education and 
are preparing to enter the labor market, those who have already 
entered the labor market but need to upgrade their skills or 
learn new ones, those with special educational handicaps, 
and those in post-secondary schools--will have ready access 
to vocational training or retraining which is of hig h quality, 
which is realistic in I'ght of actual or anticipated oppor- 
tunities for gainful employment, and which is suited to their 
needs, interests, and ability to benefit from such training." 
(19,2) 

Any state desiring to receive funds from this title is required to 
establish a State Advisory Council. One of the functions of this council 
is to evaluate vocational education programs, services, and activities 
assisted under this title and publish and distribute the results. 

One of the recommendations in the 1971 Annual Evaluation Report of 

the Tennessee State Advisory Council on Vocational Education to the U.S. 

Commissioner of Education and the National Advisory Council on Vocational 

Education was: 

"The Council recommends the establishment of a more 
comprehensive evaluation plan to evaluate programs on the 
basis of quality and process along with the product and 
that the plan be implemented at the earliest feasible 
date." (1, 3) 

Tankard in the article, 'Identifying and Providing Quality Education" 

says, 

"Education is quality education to the extent that it meets 
the needs of the young people being served, to the extent 



that it helps solve their problems and fosters their optimum 
grouth and development. It is seldom a readily identifiable 
combination of learning activities measurable by qualitative 
and quantitative criteria. It is more a direction than a 
destination . . . and there is no magic point at which a program 
suddenly becomes a quality program. An analysis of the needs 
of each individual pupil and a program for meeting his unique 
needs are critical parts of any quality education program .... 
Evaluation should, c£ course, measure the changes in student 
achievement and behavior because this is the whole purpose of 
the program." (17, 331-534). 

Rajpal, in a 1969 article from the Journal of Educational Research 

says, 

"Many people have become concerned with the quality of education 
as a result of the high leirel of expenditure for public school 
instruction. This concern for quality of education in public 
schools, as well as for the expenditure of education, has created 
a need for more empirical evidence as to what, if any, relation- 
ship exists between the expenditure and the quality of education." 
(13, 57) 

In an address by Sidney P. Marland, Jr., U.S. Commissioner of Educa- 
tion, to the 1971 Convention of the National Association of Secondary School 
Principals, he states: 

"All education is career education, or should be. And all our 
efforts as educators imist be bent on preparing students either 
to become properly, usefully ein)loyed immediately upon gradu- 
ation from high school or to go on to further formal education. 
Anything else is dangerous nonsense. I propose that a universal 
goal of American education, starting now, be this: that every 
young person completing our school program at grade 12 be ready 
to enter higher education or to enter useful and rewarding employ- 
ment." (9, 5) 

*'0f those students currently in high school, only three out 
of ten will go on to academic col lege- level work. One-third 
of those will ^rop out before getting a baccalaureate degree. 
That means that eight out of ten present high school students 
should be getting occupational training of some sort. But only 
about two of those eight students are, in fact, getting such 
training. Consequently, half our high school students, a total 
of approximately 1,500,000 a year are being offered what amounts 
to irrelevant, general educational pap!" (9, 7) 



This formal definition of program evaluation is set forth in Guide to 

Improving Vocational Education Evaluation: 

"Program evaluation is the continuous process of collecting 
valid and reliable data for the purpose of comparing program 
outcomes with program objectives. The process is conducted 
to provide useful information for making sound educational 
decisions. Educational decisions refer to making a choice 
among alternatives for action in response to educational 
needs and limited resources." (11, 1) 

Moss defines program evaluation as 

"the process of attributing differences between actual and com- 
parative outcomes to program characteristics, under different 
conditions of student characteristics and other intervening 
influences, and making a judgment about the value of the 
program characteristics. The process is conducted for the 
purpose of making more rational decisions about programs." (10,5) 

Moss also states that the justification for his paper on the evaluation 

of occupational programs rests upon the validity of three assumptions: 

"Program evaluation is essential to systematic improvement 
in educational efficiency and effectiveness; an intensification 
of evaluative activity is highly desirable; much of what little 
has been done to date in the name of program evaluat-^o'^. is of 
questionable usefulness." (10, 1) 

One of the more ctnnprehensive recent studies in evaluation of voca- 
tional education was conducted by Harold Starr at Ohio State. He sum- 
marizes the following: 

"The evaluation methodology which has been used most by state 
divisions of vocational education is process evaluation in 
which organizational structures, education processes, equipment, 
and facilities are judged against preset standards. This type 
of evaluation has been used extensively for many purposes, 
including the allocation of state resources to local programs and 
the development of budget requests to the legislatures. Assess- • 
ments of program adequacy are determined through judgmental 
procedures and the opinions of experts; and programs meeting 
standards are assumed to produce effective outcomes. 

The review of process evaluation systems reveals that they do not 
provide quantitative information required for state level manage- 
ment functions. This evaluation methodology also has certain 
practical and methodological difficulties which reduce overall 
evaluation system efficiency. These methods require extensive 
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local school visitation by state level personnel to secure 
evaluation data about programs, equipment and facilities. 
Such visitations require sizable investments in manhours and 
personnel so that it ic both difficult and expensive for any 
state to conduct frequent evaluations. 

Another approach to evaluation of vocational programs has 
enjoyed popularity. In this approach, data arc colJected and 
stored which appear to have logical relevance to decision- 
mailing requirements. No systematic attempt is made to define 
data needs in ad\^a^ce by reference to snocific program planning 
or accountability requirements. Instead, data are accumulated 
in great variety v/ith the expectation thcit any data needed 
later will probably be available in the b?nk." (15, 4-.S) 

The project staff concluded that an evaluation methodology which 
has greater payoff for state vocational education agency program 
planners in terms of program planning and accountability p^T'poses 
would be a methodology which is consistent with a systems approach to 
planning, contains process elements, but is oriented primarily toward 
product or outcome measures. (15, 11) 

Byram's report lists three general strategies for bringing about 
evaluations of local programs of vocational education— a state initiated 
and/or state operated evaluation, a state-led evaluation, or an independent 
local program evaluation. His manual focuses on locally directed evaluation 
and a systematic approach to be used. He feels strongly that an evaluation 
ought to be done by those most closely identified with the program and 
responsible for improvement of the programs. The advantage of a locally 
directed evaluation is that the persons involved are those who must sub- 
sequently implement recommendations. However, he points out that the people 
who operate the program may be so close to it that they sometimes fail to sense 
some of the important elements to be examined or the significant needs to 
be explored. Some educators feel that they lack necessary competence in 
• developing evaluation instruments and/or in analyzing and interpreting 
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the data to be collected. (6) 
Coster concludes: 

"It is highly desirable that the involvement of the potential 
employers of the products of occupational education programs 
be continued as further action is taken to improve the quality 
and quantity of occupational education programs.'* (7,10) 

Swanson's "Nationwide Study of the Administration of Vocational- 

Technical Education at the State Level*' generalizes: 

•'During recent years the urgent need for a high-quality 
program of education for ?J1 children and youth has become 
recognized. Much progress hfis been made in improving the 
status and quality of education at all levels. However, this 
progress has been uneven, with perhaps the greatest improvement 
in areas such as science, mathematics and foreign languages. 
There is a serious question whether the progress in vocational- 
technical education has kept pace in many parts of the nation 
and whether. current policies, programs ?vnd procedures are 
adequate to meet present and emerging needs in our rapidly 
changing economy." 

"On the basis of the limited evidence available from other 
studies and reports, a reasonable assumption seemed to be that 
in many states, the state educational agencies were better 
oriented to the continuatirn of traditional programs than 
to the identification, stimulation and implementation of changes 
designed to meet emerging needs of a rapidly ch?r»ging economy and 
socio-cultural setting^. (16, 197) 

An Arkansas study points out: 

"Before existing programs can be 'improved' and before access to 
programs of 'high quality' can be insured, adequate systems 
and techniques of evaluation must be developed and implemented. 
The use of quick and often highly subjective devices for appraising 
the quantity, quality, and cost-effectiveness of vocational 
programs will not suffice. Educators are gradually recognizing the 
importance and complexity of the evaluation process but have 
not yet taken the necessary steps to fully develop and operationalizc 
effective evaluation programs.*' (11, 1) 

The Pennsylvania Department of Ldncation has developed and is 
implementing a plan to assess the quality of education in the Common- 
wealth. Ten goals form the basis for the assessment. In initial development 
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of the study, data were collected from pupils in two grades across the 
state. The instrument used consisted of multiple items which were analyzed 
using a Likert-type scale. This provides an even interval measurement 
scheme similar to the instruments used in this study. A statewide sampling 
procedure was implemented whereby representative sections of the state 
were included in the sample. (2) 

Mary evaluative techniques have historically been used in securing 
data for decision making in relation to all facets of education including 
vocational-technical education. A specific list with brief discussion 
of each type is provided in a 1968 seminar paper by Arthur Bruhns. (3) 
Among those techniques he lists are: graduate interviews, career follow- 
up, achievement testing, state and national licepsing examinations, industry 
advisory committees, systems approach cycle, accreditation, self-initiated 
evaluation, personality change evaluation, and evaluative criteria* 

An explanation of each of these is included in the report but is too ^ 
voluminous to be reviewed in this section. !tov;evcr, the proup of items 
cover a broad range of evaluative techniques and present a consensus of 
those evaluative devices used in education. Many of them involve an 
interaction between the evaluator and th'^ field personnel and involve both 
direct and indirect contact (e.g., the use of a mailed questionnaire as 
well as the interview technique). 

Samples of instruments including material on program objectives and 
goal statements from such studies as the Starr report (J 5) were used in 
developing instrument format. The Evaluative Criteria (8), National 
Study of Secondary School Evaluation, also provided a source of sample 
items. Broad-based goals such as the ones from the Pennsylvania study 



were utili7ed by the research team in uTiting its components and measure- 
ment iters* (2) 

Additional items for study 'were provided frcn a si'^'V;* on ''Locally 
Directed Evaluation of Local Vocational Education Programs/* (6) 
Another source of data for instrument construction was fo^md in the forms 
used in evaluating the Institute for Improving ^'ccational Education 
Evaluation at the University of Arkansas. 

The Program Analysis Qnestionj.airn for Vocational-Technical Education 
developed as a part of Rhode Island's plan for expansion and improve- 
ment of vocational-technical programs throughout the state consisted of 
four areas- -administration and guidance, curriculum, physical facilities, 
and instruction. The plan of evaluation invoJ^-'^H the use of self -ana lysis, 
analysis by an evaluation team, and analysi3 by representatives from 
industry. (12) 

In developing the definition, list of components, and items for the 
measurement instrument, extensive use was made of materials from each of 
the six program areas such as course outlines, evaluative criteria, 
courses of study, Iiandbooks, program guidelines, goal statcronts and 
program area objectives. Several recorded state evaluation efforts were 
reviewed. Research personnel from the Tennessee RCU provided thorough, 
current listings and/or copies of relevant publications. 

A summary of the review of related literature indicates a consensus 
need for evaluation, for making that evaluation locally oriented as well 
as state and nationally oriented, and for involvement of multiple groups 
in evaluation. Various methods of evaluation are being used, including 
general checklists and questionnaires, interviews, follov;-up analyses, 
and standardized test results. The purpose of these analyses is directed 



22 

at program i.nprovor.ent and revision and each has as its undorgirding theme 
the Croat ior. of a quality proprnn. 
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SECTION III 

MALYSIS OF DATA AMD DISCUSSION OF 
PROPOSED MODEL FOR EVALUATION 

The plan for the analysis of the data collected as part of the study 
of quality vocational-technical programs has several facets. The initial 
analyses are concerned with the data gathered from the ^'Components for 
Quality Vocational-Technical Programs" checklist (Appendix B) . As was 
previously Sv.c;ted, these data were gathered from principals, guidance 
counselors, and vocatio^^-^l- technical teachers across the state, representing 
Agricultural Er-ucation, Dt stjibi'tivc Iducation, J'calth Occupations, Home 
Economics, Office Occupations, and Trac^-e and Industrial programs. The 20 
components in the ''Quality Vocational-Technical P3xcrams" checklist con- 
stituted those elements which were considered to be components of a quality 
program, and participants tvere asked to respond with an indication of the 
importance they placed on each component. Response possibilities were: 
"Essential,'^ ''Highly Desirable," "Desirable," "Of Some Importance," "Of No 
Importance," and "No Opinion." The resulting analyses of these responses 
are included in two sections. 

The first analysis in concerned with statistical manipulation of the 
data from Phase I and Phase II, v;hile thd second anrJysis is a presentation 
of frequencies and percentages of responses from Phase 1 and Phase II for 
the total group and for each prnf;ri'm area. 

Initial analysis of the data consisted of a detornination of which 
components were identified as "Essential," "Highly Desirable," etc. by the 
participants of Phase I of the study, in total and in each program area. 

The values assigned to each component for statistical purposes were: 



24 

"Essential" = 5, "Highly Desirable" ^ 4, "Desirable" ^ 3, "Of Some 
Importance" » 2, "Of No Importance" = 1, and "No Opinion" = 0. The 
responses of the three professional groups were tabulated and averaged 
for each of the 20 components for use as a method of ranking. All averages 
were computed without responses of "No Opinion." No specific cutoff value 
was assigned for any ranking, but natural breaks in the set of averages 
were selected. It was hypothesized that all components were important 
and that no artificial cutoff score would produce a more viable group of 
components than natural data breaks. 
"Quality" Component Analysis 

When an analysis of the responses from all principals, teachers, and 
guidance counselors used in the study was made. Components 1, 2, 6, 14, and 
15 were defined, by the average value of their responses, to be "Essential" 
in importance as components of quality vocational- technical programs. 
Those components judged as "Highly Desirable" were 3, 5, 7, 8, 10, 11, 12, 
16, 17, 19, and 20. As previously defined, these were components which by 
the average response from all of the teachers, principals, and guidance 
counselors in the study produced a mean sufficiently large for that component 
to be considered "Highly Desirable." Those components which the entire 
group ranked as "Desirable " again based on their average response score, 
were 4, 9, 13, and 18. No components were given values low enough to be 
ranked "Of Some Importance," "Of No Importance, •* or "No Opinion." 

Table 2 presents a composite view of the average score of each of the 
20 components of quality vocational -technical programs as viewed by the 
overall group and by each of the program areas. The table also shows, for 
example, that Component 2 received the highest average score; that 15 was 
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second, etc. hTiile the same items were defined as "Essential" by the 
Agricultural Education participants' responses, the order was different. 
When Distributive Education participant responpes were examined, the same 
components were labeled as "Essential" with the addition of Component 20. 
Mor-> specifically, the Distributive Education results designated the five 
components used by the composite group and added Component 20 as "Essential" 
for "quality" Distributive Education programs. Health Occupations par- 
ticipants also selected the initial list of 2, 6, 14, 15, and 1 for their 
responses to "Essential" components. Home Economics participants indicated 
that Components 1, 2, 6, 14, and 15 were "Essential" items but indicated 
that Component 12 with a mean score of 4.52 also was an "Essential" component 
in their program. Office Occupations responses again showed the five 
components 1, 2, 6, 14, and 15 as "Essential," as did the Trade and Industry 
responses. 

Those components which were deemed "Highly Desirable" from the overall 
responses and by the responses from the six program areas may also be found 
in Table A. These data indicate a parallel consideration among the groups, 
with three groups showing a difference in components which were "Highly 
Desirable" and those which were "Desirable," while the other three groups 
indicated that all components liad a "Highly Desirable" status. The mean 
responses produced borderline differences in several instances with the 
cutoff point being approximately 4.5 between "Essential" and "Highly 
Desirable" and approximately 3.5 betvveen "Desirable" and "Highly Desirable." 

These differences, of course, represent relatively slight variances 
and show that major agreemmt was found for responses across all service areas 
of vocational-technical education personnel. (This was expected since only 



27 

items rated highest and mentioned most often by outstanding persons in 
vocational-technical education were included on the checklist.) The data 
are interesting in several instances as they show the highest single rank 
given a component was 4.86 given to question 15 by Home Economics respon- 
dents, while the lowest mean was the 2.83 assigned by Distributive Educa- 
tion personnel responses to Component 18. The clustering of the components 
is relatively similar with only minor differences throughout the areas for 
comparison. These data will be used throughout the remainder of the analysis. 

The purpose of Phase II of the study was to allow a sample of the 
participants in Phase I to rate their local vocational-technical programs 
using the 60-item "Self Evaluation for Vocational-Technical Programs'' check- 
list. As was discussed in a preceding section, the 60-item "Self Evaluation" 
allowed for rating of program elements based on the previously identified 20 
components of a quality program. Each participant was asked to irate frankly 
his or her vocational -technical program using the following scale: 

5 - Excellent; very well done 

4 - Satisfactory; adequate 

3 - Some improvement needed; no 
crucial weakness (es) 

2 - Major improvements needed 

1 - No effort made 

0 - No opportunity to observe; 
not applicable. 

The three professional groups—teachers, principals, and guidance 
counselors— were asked to respond to the instrument in relation to a 
particular program in their school. This instrument differed slightly from 
the 20-component checklist which asked the teacher to rank the components 
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for a quality program* As has been previously stated, three items for each 
of the 20 components were developed to measure the particular program being 
evaluated by professionals and students. These items were field tested in 
East Tennessee school systems for content validity* Each respondent was 
asked to respond to each item on a one. to five scale. As in the previous 
analysis, the mean of the items were computed using the values 5, 4, 3, 2, 1 
to determine an average for each particular item. Further analysis using 
groups of three for catcjjorixation purposes will be presented later. 

Table 3 shows the means for each rcrpondpnt rroup for the 60 items. 
Tables 4 and 5 show the same data categorized by each o" ^ '^r. six program 
areas. Table 4 sliows the overall responses for each area for all respondent 
groups combined, while Table 5 givec the same information with the student 
totals excluded. 
Self Evaluation Analysis 

When the responses to the 60 items were analyzed in terms of the 20- 
component categorization of "Essential," "Highly Desirable," and "Desirable," 
Table 6 shows the mean of the "Essentia V items to be 4.05, the mean of the 
"Highly Desirable" items to be 3.78, and the mean of the "Desirable" items to 
be 3.42 for the total group of 18 programs* This table also shows the mean 
for each categorization for each program area. These data show the highest 
"Essential" mean was 4.25, given by the Health Occupations respondents; and 
the lowest "Desirable" score, of those groups who had "Desirable" elements, 
was 2.76 from the Agricultural Education respondents. The data show the 
pattern for "Essential" items to be responses over 4.0 in each case except 
two, and 3.9 and 3*5 in those two cases. Comparable data from student 
responses (Table 7) included in this particular facet of the study showed an 
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TABLE 6 



MEAN VALUES FOR ESSENTIAL, HIGHLY DESIRABLE AND 
DESIRABLE COMPONENTS FROM PROFESSIONALS' RESPONSES 
BY TOTAL GROUP AND BY PROGRAM AREA 



Component 


Total 


Ag. 






Hone 


Office 




Categorization 


Group 


Ed. 


D.E. 


Health 


Ec. 


Occ. 


TSI 


Essential 


4.05 


3.53 


4.15 


4.25 


4.23 


4.14 


3.93 


Yighly Desirable 


5.73 


3.04 


3.88 


4.20 


3.85 


3.77 


3.56 


Desirable 


3.42 


2.76 


3.43 


3.42 









TABLE 7 



MEAN VALUES FOR ESSENTIAL, HIGHLY DESIRABLE AND 
DESIRABLE COMPONENTS FROM STUDENTS' RESPONSES BY TOTAL 
GROUP AND BY PROGRAM AREA 



Component 


Total 


Ag. 






Home 


Office 




Categorization' 


oroup 


Ed. 


D.E. 


Health 


Ec. 


Occ. 


T5I 


Essential 


3.83 


3.96 


3.51 


4.12 


3.68 


3.89 


3.94 


Highly Desirable 


3,62 


3.72 


3,28 


4.00 


3.46 


3.60 


3.55 


Desirable 


3.61 


3.60 


2.88 


3.31 









ERIC 



60 

overall mean of 3.83 for those items which were judged "Essential'* by the 
professional groups, a mean of 3.62 for the ''Highly Desirable" items, and 
a mean of 3.61 for the "Desirable" items. The data indicate that average 
responses from students were somewhat more clustered than were the 
responses from the professionals. The differences, especially at the top, 
were relatively minor. The responses of the Health Occupations students 
on the "Essential" items show the highest single response average of 4.12, 
v/hich paralleled the hinhest response from the professionals. 
Model Development 

In devising a plan to be used for analysis, it v;as decided to analyze 
the three category groups based on the responses to the 60 items as a 
function of whether the sets of it«ns represented an "Essential" component, 
a "Highly Desirable" component, or a "Desirable" component. The ranges 
given in Table 8 for each set of 3 items (i.e., 1-3 measuring Component 1; 
4-6 measuring Component 2, 7 - 9 measuring Component 3, etc.) was computed 
to determine a measure of agreement among sets of items. This data could 
provide input for possible revision of selected items. Table 9 presents 
comparable rara from student responses. 

It is interesting to note that the overall average variability for 
the ^:ioups of thre'^ iter^s corresponding to the con^:onents termed "Essential" 
was 0.39, for those termed "Highly Desirable" was 0.61, and for those termed 
•'Desirable" was 0.72. The scores indicate that evaluations tended to be 
moie hcmogejieous with an increase in the desirability of a component. Further 
emphasis of this homogeneity is .given by the occurrence of the six highest 
variations for the overall group in the ten lowest ranked questions. See 
Table 3. 
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The data in Table 2 and Table 8 for the overall group is compared 
graphically in Figure 1. This figure illustrates the degree to which 
each component is necessary in a "quality" program and the associated 
evaluation score for each of the three items corresponding to a given 
component. The darkened areas show the relative success each of the 
items in the set of three has in measuring the specific component under 
question. For example, the degree to which Items 4, 5, and 6 measure 
Component 2 can be told from a study of the degree of difference in location 
of the solid line and the broken line for Component 2. The shaded area 
shows the relationship for each item (4, 5, or 6). It is then obvious 
that the previously determined "Essentiar' components in addition to 
Component 12, which was identified as "Essential" in two of the six areas, 
show a greater difference between their desirability and the present state of 
accomplishment. It is also obvious from Figure 1 that the majority of 
the components were presently being met with more than a rating of 
"Some Improvements Needed," but not quite a rating of "Satisfactory." 
The sharp drop in Component 4 (Item 10) can be explained by the nature 
of the question; home visitation is classified as "Desirable," while 
in actual practice little is done except in the areas of Agricultural 
Education and Home Economics. 

Since one of the major goals of this study was the production of a 
usable evaluation instrument for all levels of personnel in vocational- 
technical programs, a non-complicated model was prepared using the results 
of th: i section. The following section illustrates the use of this 
model in program evaluation. 
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In Figure 2, the items (for the overall responses) are grouped 
according to the previously defined classification scheme and weighted by 
multiplying those classified as ^'Essential*' by 30, those termed "Highly 
Desirable" by 20, and those termed "Desirable" by 10. This weighting 
was simply to increase the visibility of differences in t' e grouping 
pattern in the Figure. The "Self Evaluation for Vocational-Technical 
Program" instrument (60-it«n checklist) will he used to evaluate a 
vocational -technical program in the following manner. The Self Evaluation 
Instrument could be utilized by the principal, teachers, and/or students 
of a particular school or used by an outside evaluator. The results can 
then be averaged in the manner previously discussed with the ratings for 
the 60 items grouped by three's and averaged for comparison with the 
results from the composite (Figure 2). Table 10 presents the weighted 
responses for each component for the overall group and for the responses 
by program area. For example, the weighted average for Component 1 was 
139.7, This may be seen as the value representing this component on Figure 2. 

The significant factor in analyzing the status of a program would then 
be the deviation between the established component results and the three- 
item averages, A deviation is obtained by the formula D^ = Ai - Ri, where 
Ri is the weighted i_-th established result from Table 10 where ^ is the number 
of any component A^^ is the weighted correb^onding three-iton average 
computed from local data. The value of this computation (D^) shows whether 
the school program is better than, equal to, or worse than the norm program 
on this component. More specifically, i^ represents the number of any component 
(1 - 20), Ri represents the norm or sample response average for the three 
items measuring that component and Ai is the average for the three items 
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from the local program being studied. is then the difference in these 
two scores. 

For example, if i = 2, R2 is, using Table 10, shown to be 141.5, and 
if A2 were 132.0, the resulting deviation D2 would be -9,5. This would 
indicate that some improvement is needed on this component. On the other 
hand, if is greater than or equal to Rj , then the school can be considered 
as having a successful program as measured by this instrument on that par- 
ticular component. It is important also to analyze the magnitude of the 
negative deviations in terms of the characterizations into "Essential," 
"Highly Desirable," and "DcFirablc'- components. Figure 2 shows that the 
magnitude of one negative ccr.poncnt for an ^'I-ssential" item will not compare, 
in general, with those from the other item?. For exampJc, a reaction to a 
deviation of -9.5 on an "Essential" component u'ould net be the same as a 
reaction to a deviation of -9.5 for a "Desirable" component. Thus, this 
instrument will locate thcne components in which improvement is needed; 
and it will provide information as to degree of change possibly needed. 

It should be strcssrifl that the magnitude of the deviations are only 
an indication of amount or degree of improvement needed within a program 
area. Obviously as the data in Table 10 show, there are slight differences 
among program areas when compared with each other or the overall results. 
Therefore, the application of current procedures will be an individual 
decision based on the data included in this report to provide a basis 
from which to inii:iate program revisions. 

Figures 0-8 provide comparable pictorial views of the weighted 
responses and their plots for "Essential," "Highly Desirable," and "Desirable" 
elements by program area. The application of these data in program comparisons 
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would be made in the same manner in which the overall comparisons are 
made. It should be recognized that fewer persons, of course, responded 
for e -ch area than is represented by the overall responses. Individual 
program area comparisons would be made in the manner similar to that for 
the overall data. 
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SECTION IV 

Ar:ALYSES OF COMPONENTS AND 
SPLF-EVALUATION RESPONSES 

The second major facet of the analyses of the data fro«n the responses 
of professionals to "Components of Quality Vocational-Technical Programs" 
and of professionals and students to the "Self-Evaluation" checklist is a 
presentation of frequencies and percentages by various groupings. 
P rincipals* Responses to "'Co mponent s" 

In general, the strongest consensus was among the principals. At least 
three-fourths of the SO principals responding indicated that all but six of 
the components were either "Essential" or "Highly Desirable" in quality 
vocational- technical programs. Nearly all of the other respondents to the 
six remaining components (4, S, 9, 13, 17, 18), felt these components to be 
at least "Desirable." A small number indicated some components to be "Ot 
No Importance" or had "No Opinion." Likewise, a relatively small number 
felt that several- components were "Of Some Importance." 

As with teachers and counselors. Components 1 and 2 were considered 
"Essential" by the largest proportion of principals (over 70 percent). 
Component 18 was felt to be "'Of Some Importance" of "Of No Importance" 
by a relatively large nunber (T4 percent) . This was the only item ranked 
low by more than a token number. These data appear as Table 11. 
Teachers' Responses to "Components" 

At least three-fourths of the 50 teachers responding agr.ed that most 
components were "tjsential" or "Highly Desirable" with the exception of 
Component 4 (parent-student-teacher conferences). Component 5 (flexibility for 
student movement within the program). Component 9 (use of advisory committees). 
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Component 13 (leadership development programs), Component 17 (placement 
of students in occupational "clusters"), and Component 18 (placement 
services). Unlike principals' responses, but similar to guidance counselors' 
responses. Component 10 (follow-up surveys) and Component 16 (provide 
for 90 percent student completion) were ranked low. 

As with guidance counselors, nearly all of the remainder felt the 
component to be at least "Desirable.'' Generally only a token number 
indicated the component to be "Of No Importance" or had "No Opinion." 
A small number indicated a few of the components to be "Of Some Importance." 

As with guidance counselors' and principals' responses. Component 18 was 
ranked low by more teachers than any other component. More than 75 percent 
of the teachers considered Components 1. 2, and 6 as well as Component 14 
(proper attitudes and work habits) and Component 15 (total student develop- 
ment) "Essential." These data appear as Tallc 12. 
Guidance Counselors' Responses to "Components" 

The counselors' responses, like thosf, of principals and teacKers, 
tenc:d to follow a similar pattern. Thzre were, of course, some differences. 
Only half of the components were cor.sidered "Essential" or '^Highly Desirable" 
by more than 75 percent of the 53 counselors responding, although almost all 
(generally more than 90 percent) agreed chat each component was at least 
"Desirable." 

Surprisingly, ^ierhaps. a ntuiilier of guidance counselors did not consider 
Component 3 (testing and guidance programs). Component 10' (follow-up surveys 
and up-to-date records), or Component 18 (placement services) "Essential" — 
these are traditionally within the domain of guidance services. One might 
expect grep.ter commonalty of responses from guidance counselors on these 
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components. 

Guidance counselors tended in their responses to lean more toward 
the principals' point of view than the teachers \ For example, 90 percent 
of the teachers considered Component 7 (certified teachers) to be 
"Essential" or "Highly Desirable?; 80 percent of counselors and 78 percent 
of principals responded similarly. While these differences among groups 
were slight, they were noticeable. 

More than 70 percent of the guidance counselors considered Component 1 
(complement program objectives). Component 2 (support of administrators), 
and Component 6 (facilities, equipnent and materials) to be "Essential." 
These data appear as Table 13. 
Principals' Responses to "Self-Evaluation" 

Principals tended generally to rate their programs on the high side. 
There were, however, some exceptions. Response patterns were similar in 
many ways to those of teachers and-counselors, yet differences were 
noticeable. 

Over half of the principals gave an "Excellent" rating to Items 2, 
37, 40, 45, and 47. In agreement with the counselors, no principal gave 
Item 10 an "Excellent" rating. 

As noted above, more than half of the principals (with some exceptions) 
rated their programsas either "Excellent" or "Satisfactory"; in many cases 
the percentage was much greater than half. In general agreement with 
counselors and teachers, more than SO percent of the principals assigned 
lower ranks to Items 10, 11, 12, 2S, 26, 27, 31, 3S, 48, 49, and 56, thus 
indicating that these are generally accepted as areas of need by the three 
groups. 
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The only area seen by more than a token number of principals (over 
20 percent) in which *'No Effort" had been made was Item 10 (home visits) . 
Items 11, 23, and 26 were the only ones seen as "Needing Major Improve- 
ments" by more than a fifth of the principals. Thus principals, by a very slim 
margin, were the least inclined group to assign very low ratings to it«ns. 
These data appear in Table 14, 
Teachers' Responses to "Self -Evaluation" 

Almost all teachers (83 percent) rated Item 38 as "Excellent"--almost 
two-thirds of the counselors had done the same— while at least half of the 
teachers gave an "Excellent" rating to Item 32 (curriculum content). Item 40 
(evaluation of students). Item 45 (ethical practices), and Item 55 (favorable 
community reputation). No teacher ranked Item 9 (review of tests). Item 12 
(parent conferences), or Item 51 (occupational sureness) as "Excellent," 

Item 44 (vocational student's self image and achievement) was rated 
"Satisfactory" by 83 percent of the teachers while this rating was assigned 
by more than half of the teachers to an additional twelve itons: 1, 5, 14, 
15, 21, 22, 23, 36, 41, 42, 50, and 56. Only in Items 14 and 15 (making 
changes in the student's program) v/ere simiJar percentages of counselors 
in agreement with these teachers. 

As did the counselors, more than half of the teachers ranked most 
aspects of their programs "Satisfactory" or "Excellent." As with counselors, 
the exceptions included Items 10, 11, 12, 26, 27, 31, 35, 48, and 52. Unlike 
counselors, more than half of the teachers also gave low ratings to Items 
18, 59, and 60, 

^fore than a fifth of the teachers indicated that "No Effort" was made ^ 
in Item 10 (home visits) as well as Item 12 (parent-teacher conferences) and 
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Item 59 (occupational information) . More than a fifth indicated '^Major 
Improvements Needed" in Item 6 (areas of resources). Items 11-12 (parental 
involvement). Item 16 (tools and supplies), item 27 (advisory committee 
assistance). Item 31 (occupational survey). Item 48 (factors preventing 
the completion of programs), and Item 60 (resource persons). 

Generally, the teachers— like the counselors— did not identify many 
areas needing major improvements. Although the ratings were divided in 
many cases, the trend with teachers was toward adequate or better ratings. 
These data appear as Table 15. 

Guidance Counselors' Responses to "Self -Evaluation" 

At least half of the counselors rated their vocational -technical 
programs "Excellent" in only two areas: I,tem 5 (interaction among adminis- 
trators, etc.) and Item 38 (rrembership in vocational youth organizations). 
Also half or more of the counselors rated their programs "Satisfactory" in only 
six areas: Item 14 (changes in student's program). Item 15 (assistance in 
program changes). Item 19 (good community relations). Item 30 (follow-up 
requests). Item 41 (student pride in worlonanship) , and Item 54 (sources for 
entry-level jobs). With the exception of Item 18 (adequate working and 
storage space), which half of the counselors rated "Some Improvement 
Needed," no other item was agreed upon by as large a group. No counselor 
rated Item 10 (home visits) as "Excellent." 

With some exceptions, more than half of the counselors rated their 
programsas either "Excellent" or "Satisfactory"; areas not so ranked by as 
large a group included Items 10-12 (parental involvement). Item 23 (training 
plans for occupational experience). Items 25-27 (advisory committees), 
Itm 31 (annual occupational survey). Items 35-36 (program evaluation), 
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Item 48 (factors preventing completion of programs). Item 52 (placement 
service), and Items 56-57 (citizen knowledge and support). 

Only in Item 10 (home visits) did a relatively large number (22 percent) 
of counselors feel that there was "No Effort Made." A similar number felt 
there were "Major Improvements Needed" in Items 8, 10, 31, 48, and 52. 

Kith these exceptions, a high proportion of counselors generally did 
not identify a large number of areas in which "No Effort" was made or "Major 
Improvements" ivere needed. In a few areas about one-fourth identified 
areas where "Some Improvements" ivere needed, but these were generally offset 
by larger numbers agreeing that the area was "Satisfactory" or "Excellent." 
These data appear as Table 16. 
Students^ Responses to "Self -Evaluation" 

Students' responses generally fell into a pattern similar to that of 
guidance counselors, teachers, and principals. Since the student sample 
was over ten times as large as any of the other three groups, this 
generally supports the accuracy of the smaller groups perceptions. 

Students were more likely to assign a rating of "Some Improvement Needed" 
to many items than were principals; however, so did teachers and counselors. 
As with the other groups, a large proportion (approaching or exceeding 
half) of the students assigned high ("Satisfactory" or "Excellent") ratings 
to most items. Students identified areas which needed improvement that 
had also been identified by the other groups. 

A careful comparison of student and teacher responses indicated that 
while some students generally did not rate each item as high as teachers, 
with the exception of Items 30, 32, 34, 35, 37, and 48, the percentage of 
teachers and students rating items in ^hc same way was remarkably similar. 
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These data appear as Table 17. 
Additional Analyses 

Additional tabular presentations of the 20 components of "quality** 
programs and the ''Self-Evaluation*' checklist were made. These data include 
frequencies and percentages by program area and by respondent group and 
mean values by program area and respondent group. Tabular presentations of 
this material appear as Appendix I. A detailed list of these tables may 
be found under ^pendices in the Table of Contents. 
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SFXTION V 
GHNHRAL PROGRAM SURVEY 

As outlined in the Procedures section, a general information question- 
naire was completed by the teachers during the site visit or immediately 
after the conclusion of the visit, (See Appendix G) . These data were 
sought to provide a more detailed look at several aspects of the program 
related to the component?; for ^'quality'' vocational -technical programs and 
to facilitate future instrument revision and expansion. 

In general, th3 18 schrolr either have a wide range of vocational- 
technical areas represented or offer, for examrie, only home economics and/or 
agricultural education. All teacherr, surveyed, representing the range 
of vocational courses, provided similar answers to a wide range of questions. 
These results indicated that: (1) all teachers encourage parents to visit 
the schools; (2) teachers are member* of professional organizations; 
(3) most programs liave an advisory council; (^) all programs have a youth 
organization which the program area teacher generally sponsors; and (5) 
programs generally havo community support. 

Several questions pertained to rtirrjculum development, and some weak- 
nesses were indicated by the teachers. First, progra*^ objectives exist 
for most of the programs and have been revised within the past two years. 
However, enough teachers indicated that either no objectives had been 
formulated or those in existence had not been revised in some time to warrant 
concern in this area. Major curriculum concerns included the testing 
programs, program evaluation, and program revisions. 

Only about one-half of the teachers indicated that a testing program 
was in effect in their schools. Even these reported that the results were 
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seldom analyzed in conferences with the parents and students. Another 
, indication that testing is not adequate is evident from the fact that some 
teachers specifically indicated that guidp.nce counselors, although closoly 
involved with testing, did not actively support the vocational programs 
Furthermore, few if any parent-teacher joint conferences were held before 
a student entered a specific program and afterv/ard only. where a problem 
aroso. Joint conferences v;ith the student-coord inator-employer were 
indicated both on a schrjc^'^"' or' banis an^- on p,r> '^jf needed'' basis. 

Program evaluations c^ner-ed to he prevalent in the vocational -technical 
areas. In general, the teachers evaluate th^-'^ programs in the 
form of biannual written reports, F.valuations of the occupational aspects 
of the programs also wc^'e Jonc through writvcn reports in soipe cases, while 
others used employer-teacher conferences, Most programs use follow-up 
surveys v/hich, in general, do not include dropouts. Returns on the 
follow-up surveys ranged from 20 percent to C5 percent, but only "sometimes" 
were the results used to revise the program. The teachers did report that 
50 percent or more of the students complete their programs, 50 percent or 
more took jobs in tl:e "cluster" for v;hich they were trained, and about 50 
percent actually were employed where they got their occupational experience. 

Only some of the programs had a placement service. Finally, in the 
overall evaluation of their programs, the teacners felt it important that 
a positive change in the students' attitudes was observed. 

It has been noted that '*^esults of the follow-up surveys were generally 
not used specifically for curriculum revision. Most teachers also surveyed 
the local employment opportunities periodically, hut again little applica- 
tion v;as made of these results in terms of revising the curriculum. 
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The vocational-technical teachers can be described as a cooperative, 
well-trained, p^^ofessional group. They confer and plan with the general 
education teachers; they cooperate with each other in placing students in 
their programs; they spend time preparing instructional materials for their 
classes; and they participate in their professional organizations. A large 
number bave actual work export onr.e in the areas in which they are certified. 
A majority of the teachers indicated satisfaction with their salaries, 
while a few were dissatisfied because of different lengths of contracts 
and the salaries of uncertified persons in similar positions. 

Another point of dissatisfaction was the shortage of space in some of 
the schools. The vocational-technical programs were found to be open to 75 
percent or more of the applicants depending on the rrea involved. As 
previously indicated, the students have the vocational-technical teachers' 
guidance in selecting the proper courses; and, when changes are necessary 
and the program permits, changes are allowed. Students are acquainted with 
the programs through vocational fairs, career tours, guidance counselor 
conferences, field trips, group discussions, and films. The general public 
is kept informed through newspaper articles, radio reports, local community 
groups, business groups, parents, and reports from students. 



SECTION VI 
SUMMARY, CONCLUSIONS AND REC0^^5^M^ATIONS 
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Summary 

Preliminary investigations indicated an inadequacy of material which 
could be used by teachers, administrators, and supervisory personnel in 
determining and measuring the quality of vocational -technical programs. 

The basic purpose of this study was to establish a model for evalu- 
ation of the quality of secondary vocational -technical programs in the 
State of Tennessee. Tv;o specific objectives of the study were (1) to 
determine an acceptable definition of "quality" in vocational-technical 
education and (2) to test the workability of this definition in selected 
secondary vocational-technical programs across the state. 

As was explained in the Procedures section of this report, related 
literature was used to develop a tentative definition for "quality" 
secondary vocational-technical programs. Concurrently, 20 of tho most 
important components of a quality vocational -technical program were sum- 
marized. Knowing this was a library research effort at this point and lacked 
validity in terms of field inputs, the researcher circulated the tentative 
definition and 20 components to teacher-educators in vocational -technical 
areas at the University of Tennessee, Knoxville and to selected regional 
supervisors for review. 

There being no specific measure for quality vocational programs in the 
State of Tennessee, the research team decided to ask the regional supervisor (s) 
in each of the three main divisions of the State to identify four programs 
in his area with quality characteri5tj.cs. From these four suggestions the 
researcher drew at random three programs from each area. This sample 
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represented 18 programs in each region or nine programs in each program 
area across the State* Once an identification had been made of the programs 
to be used, a letter was ^/ritten to the principals of the 54 schools asking 
their cooperation by participating in the study. The principals were sent 
packets of material including an exr ination of the study, how they were 
identified for participation, and a request that they give other enclosed 
copies of the tentative definition and components to guidance counselors 
and teachers connected with the selected vocational -technical programs. The 
responses to this mailing served as the basis for ranking components of 
quality programs." When an analysis of the responses to the "Components for 
Quality Vocational-Technical Programs'* checklist from principals, teachers, 
and guidance counselors v;as made, it was found that five components were 
classed as "Essential"; 11 components were classed as ''Highly Desirable"; 
and 4 components were classed as "Desirable." Additional analyses were 
made by program area. 

Concurrent with the securing of responses concerning the definition and 
20 components from principals, teachers, and guidance counselors across the 
State, 60 items (three to measure each component) were developed to evaluate 
vocational-technical pronrams on the basis the 20 components. The items 
v;ere written, reviewed and revised by vocational-technical teacher-educators, 
curriculum specialists, vocational -technical doctoral students, and research 
team members. 

From the initial 54 schools one school in each program area in each 
region was selected to participate in Phase II of the study. This sample 
included only those from whom responses by all three professional groups to 
the initial mailing were received. In this phase senior-level students' 
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responses were sccurtd. T!;c prirpcs*^ th^^ p\ase of the study was to have 
a sanp]c of the participants >-nte thon- 3ccai ^'ccaticnal-technical program 
using the 60-itein instrument, "Self -Evaluation for Vocational -Technical 
Programs." 

When the responses to the 60 items were analyzed in terms of the 20- 
component categorizations of "Essential," "Highly Desirable,'' and "Desirable," 
the data showed that the mean of the "Essential" items was 4.05, the mean 
of the "Highly Desirable" items was 3.78, and the mean of the "Desirable" 
items was 3.42. 

In producing a nodel to be used ?or analysis, the researcher decided 
to analyze the three category groups based on the responses to the 60 
items as a function of their component classification. It was interesting 
to note that the overall average variability for the groups of three items 
corresponding to the ccmnonents termed "HsscntJal" was 0.39; for those termed 
"Highly Desirable," 0.61; and for those termed "Desirable, '■'o*.72. These 
scores showed that evaluations tended to be more homogeneous with an increase 
in the desirability of a component. 

Since one of the major goals of this study was the production of a usable 
evaluation instrument for all levels of personnel, a non-complicated model 
was prepared using the results of this data. The items for the overall 
responses were grouped according to the previously defined classification scheme 
and weighted by multiplying those categorized "Essential" by 30, those 
categorized "Highly Desirable" by 20, and those categorized "Desirable" by 
10. This weighting v:ar> to increase the visibility of differences in the grouping 
pattern in the figures. The "Self-Evaluation for Vocational -Technical Programs" 
instrument (60-item checklist) could be used as a part of the evaluation of 
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any vocational technical program. 

The significant factor in analyzing this program would then be the 
deviations between the established component results and the three-item 
averages. It is important also to analyze the magnitude of the deviation 
in terms of the categorization as "Essential,'' '•Highly Desirable/' and 
••Desirable" components. In this manner, the instrument will locate those 
components in which improvenent is needed. It should be stressed that the 
magnitude of a deviation is only an indication of the degree of improvement 
needed in the program area. 
Conclusions 

The research team secured agreement on a consensus definition of 
"quality" and the 20 most important components for quality vocational- 
technical programs from a representative group of vocational-technical 
personnel across the State of Tennessee. Vocational-technical personnel 
ranked the identified components as "Essential," "Highly Desirable," 
or "Desirable." 

This study showed that among vocational-technical program areas there 
is general agreement as to what constitutes components of a quality program. 

From the data it was possible to create a non-complicated model which 
can be used to assess the relative position of local programs with regard to 
those components identified as measuring quality programs. 
Rec ommendat ions 

The research team recommends the following: 

1) That the proposed model be used in evaluating a*' vocational- 
technical programs in Tennessee and that these d ii be compared 
with results of this study. 

a) Data should be gathered from principals, teachw s, and 
"senior-level" students. 
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b) Valuable input could be secured from advisory council 
members, employers, and State supervisory personnel. 

2) That graduates of vocational -technical programs be systematically 
followed up at the end of the first year out of a program and 
again after three years and asked to complete the "Self* 
Evaluation" checklist as part of a comprehensive follow-up* 

a) These data should be used to keep programs "current." 

b) Insights into possible weaknesses of vocational-technical 
programs might be provided by interviewing program "drop- 
outs." 

3) That the "Self-Evaluation" checklist might be expanded to 
include sections of items specifically designed for each 
program area. 
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DEFINITION 

A "quality" vocational-technical program will enroll an interested 
(even though disadvantaged and/or handicapped) student, assess his needs 
and abilities, and prepare him for successful entry-level employment 
(without additional training in the same skills) in the occupational 
"cluster" for which he, his parents, and the teacher have jointly 
agreed he should train. 

REACTION 
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COMPONENTS OF 
•'QUALITY" VOCATIONAI-TF.CHNICAL PROGRAMS 



All ''quality'' vocational -technical programs should: 

1. Advance, not conflict with, the objec- 
tives of the total educational program. 

2. Have support of administration (as much 
as academic programs), guidance counselors, and 
other teachers. 

3. Provide testing and vocational guidance 
services so that each student can be guided into 
a proper vocational program. 

4. Arrange parent- student- teacher con- 
ference when student is enrolled. 

5. Provide flexibility for student to 
move within program if need and/or ability 
indicates a move is advirable. 

6. Provide "adequate" physical facil- 
ities and equipment and appropriate instruc- 
tional materials. 

7. Use only teach??rs who have ccTipleted state 
certification requirements for area in which 

they teach, who have recent related work 
experience, who are proficient in skills they 
teach, and who engage in professional growth 
activities. 

8. Provide cooperative work experience 
or simulates work experience in classroom 
setting. 

9. Actively use "representative" advisory 
committees in program planning and evaluation, 

10. Conduct periodic follow-up surveys of 
former students and keep up-to-date records 
for use in program planning and evaluation. 

11. Make periodic surveys of employment 
opportunities and make needed changes to keep 
curriculum relevant. 

12. Have plan of continuing and systematic 
evaluation for improvement and development 

^ of program. 
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Essential 


Highly Desirable 1 


Desirable 


Of Some Importance 


Of No Importance 


No Opinion 


13. Endorse related vocational youth 
organizations for leadership and development. 














14. Develop student's awareness of the 
individual's responsibilities as a participating, 
productive, voting citizen in a democratic society. 














15. Develop proper attitudes toward work, 
good employer*employee relationships, and effici- 
ent work habits, and create the desire to con- 
tinue personal and occupational development. 












t 

i 
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— ...--^J 


16. Emphasize need for ethical practices 
and standards in the conduct of business. 












5 
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17. Provide for satisfactory completion of 
defined program for at least 90 percent of 
students who do not transfer or withdraw for 
health reasons. 

1 








1 

1 






18. Place majority of students in the 1 
occupational "cluster'* for which they are trained. j 
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19. Make an organized school placement ^ 
service available to graduates. ! 

20. Keep public informed - have support of \ 
community. ' 

21. Use all available means to acquaint [ 
elementary and junior high students with i 
vocational-- technical career opportunities. 
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DEFINITION OF "QUALITY" VOCATIONAL-TECHNICAL PROGRAM 



The following statement is a consensus definition of a 
"quality" vocational-technical program derived from a compre- 
hensive review of current literature. Vocational-technical teacher 
educators at the University of Tennesses, Knoxville have reacted 
to this definition. Now we' are asking for reactions from princi- 
pals, vocational teachers, and guidance personnel. We hope to 
formulate a definition that is generally acceptable for all 
vocational-technical programs in Tennessee, Is this general 
definition acceptable to you? Would it be workable in your 
program? Your comments will be most valuable to the research 
team. 



A "quality" vocational -technical program will 
provide relevant occupational information, enroll 
an interested (even though disadvantaged and/or 
handicapped) student, assess his needs and abilities, 
and prepare him for successful entry-level employ- 
ment (without additional training in the same skills) 
and place him in the occupational ''cluster" for 
which he, his parents, and the teacher have jointly 
agreed he should train. It is assumed that "quality" 
programs will always strive to develop job-keeping 
and job-advancing skills, 

COMMENTS 
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COMPONENTS FOR "QUALITY'' VOCATIONAL-TECHNICAL PROGRAMS 



Please indicate the importance you place 
on each component. 



♦'Quality'' vocational-technical programs 
should: 



1. Complement, not conflict with, the 
objectives of the total educational program 

2. Have verbal, visible support of ad- 
ministration, guidance counselors, and other 
teachers (as much as academic programs) 

3. Provide a broad-based testing 
program and vocational guidance services 
so that each ^tudent can select a proper 
vocational program 

4. Arrange parent-student-teacher 
conference when student is enrolled and 
throughout his program, if feasible 

5. Provide flexibility for student 
to move within program if need and/or 
ability indicates a move is advisable 

0. Provide physical facilities, equip- 
ment and instructional materials appropriate 
to occupational "cluster" for which student 
is being trained 

7. Use only teachers who have com- 
pleted state certification requirements for 
area in which they teach, who have recent 
related occupational experience, who are 
proficient in skills they teach, and who 
engage in professional growth activities 

8. Provide cooperative or directed 
occupational experience and/or simulated 
occupational experience 

9. Actively use "representative" ad- 
visory committees in occupational needs 
assessment, program planning, and evaluation 
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10. Conduct periodic follow- 
up surveys of former students and 
keep up-to-date-records for use in 
program planning and evaluation 

11. Make periodic surveys of 
employment opportunities and use find- 
ings to keep curriculum relevant 

12. Have plan of continuing and 
systematic internal evaluation for 
improvement and development of program 

13. Endorse and implement active 
leadership development program through 
related vocational youth organizations 

14. Develop proper attitudes toward 
work, gooJ employer-employee relation- 
ships, and efficient work habits and 
create the desire to continue personal 
and occupational development 

15. Stress the need foi ethical 
practices and standards in the conduct 
of business 

16. Provide for satisfactory com- 
pletion of defined program for at least 

90 percent of students who do not transfer 
or withdraw for health reasons 

17. Place students at the comple- 
tion of their programs in the occupational 
"cluster" for which they are trained 

18. Make an organized school place- 
ment service available to graduates 

19. Keep public informed - have 
"participatory" support of community 

20. Use any available means to 
acquaint students at all levels with voca- 
tional-technical career opportunities 
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If you feel that any necessary components have been overlooked, please 
add and rate than. *^ 
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THE UNIVERSITY OF TENNESSEE 
KNOXVILLE 37916 



1S8 



COLLEGE OF EDUCATION 

DEPARTMENT OF 
CURRICULUM AND INSTRUCTION 



March 27, 1972 



Principal 
High School 
Anywhere, Tennessee 

Dear ^fr. Principal 

A State-funded research project at the University of Tennessee, Knoxville 
is in progress to arrive at a consensus definition for "quality" in vocational- 
technical programs. The development of a model or models to facilitate 
local and State evaluations and program planning and development is another 
facet of the overall project. As you know, quality vocational -technical 
education has received even higher priority concern with the current national 
emphasis on Career Education at all grade levels. 

The research team, in cooperation with State personnel, has identified programs 
with "quality" characteristics in each region by program area. Your Distributive 
Education program has been selected to participate in this study. Our initial 
request is for you, the Distributive Education teacher (s), and the guidance 
coiinselor(s) most familiar with this program to react to the consensus definition 
of "quality" for vocational-technical programs and to complete the attached 
checklist. As we refine the instrument and develop additional checklists, 
we will ask for your continued help. We feel that the varied experiences 
of principals, vocational-technical teachers and guidance personnel will 
provide valuable input for this research. 

A copy of this letter and a stamped return envelope are attached to the 
forms to be given to the Distributive Education teacher and the guidance 
counselor. Could we please have the completed forms by April 14. 

We feel that the success f this much-needed project and the implementation 
of its results depend largely on your cooperation and support. If you have 
any questions about this study, please call me collect at 615-974-3165. 

Sincerely, 



John R. Ray 
Associate Professor 

JRR:le 

cc: Vocational-Technical Teacher 
Guidance Counselor 
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DEFINITION OF 'EQUALITY" VOCATIOMAL-TECHNICAL PROGPA^l 



The following statement is a consensus definition of a 
"quality" vocational-technical secondary program derived 
from a comprehensive review of current literature. Vocational- 
technical teacher educators at the University of Tennessee, 
Knoxville have reacted to this definition. Now we are asking 
for reactions from principals, vocational teachers, and 
guidance personnel. We plan to formulate a definition 
that is generally acccptahJe for all vocational-technical 
secondary programs in Tennessee. In the area of Home Economics, 
thii> definition applies only to Vocational Hoire Economics. Is 
this general definition acceptable to you? Would it be workable 
in your program? Your comr.cnts will be rest valuable to the 
research team. 



A ''quality'" vocational -technical secondary 
program will provide current occupational informa- 
tion, enroll an interested, qualified student (re- 
gardless of his mental or physical potential), 
assess his needs and abilities, and prepare him for 
successful entry-level ©nployment (without addi- 
tional training in the same skills)* and place 
him in the occupational "cluster'' for which ' 
his parents, the vocational teacher, guidam 
counselors, and other teachers have jointly ^reed 
he shoitld train. ^'Quality*' programs will alv;ays 
strive to satisfy basic needs and to develop job- 
getting, job-keeping, and job-advancing skills. 

*The goal might be defined to provide training 
which would prepare him for entry into post- 
secondary and/or college training in his field. x 



Revised May, 1972 
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COMPONENTS FOR "QUALITY" VOCATIONAL-TECHNICAL PROGRAMS 



Please indicate the importance you p3ace on each 
component . 



"Quality" vocational-technical programs should: 
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1. Complement, not conflict vith, the objectives 
of the total educational prcp.ran. 

2. Have verbal, visible support of administra- 
tion, guidance counselors, and ether teachers (as n^uch 
as academic programs) . 

3. Provide broad-basod testing program and 
vocational guidance services so that each student can 
select a proper vocational progr«?n. 

4. Arrange parent -strident -teacher conference 
when student is enrolled and throughout his program, 
if feasible. 

5. Provide flexibility for student to move 
within program if need and/ or ability indicates a 
move is advisable. 

6. Provide physical facilities, equipment and 
instructional materials appropriate to occupational 
"cluster" for which student is being trained. 

7. Use only teachers who have completed state 
certification requirements for area in which they 
teach, who have recent related occupational experience, 
who are proficient in skills they teach, and who 
engage in professional grov/th activities. 

8. Provide cooperative or directed occupational 
experience, and/or simulated occupational experience. 

9. Actively use "representative" advisory 
committees in occupational needs assessment, program 
planning, and evaluation. 

10,> Co' .t periodic follov/-up surveys of former 
students and Keep up-to-date records for use in program 
planning and evaluation. 
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11. Make periodic surveys of employment oppor- 
tunities and use findings to keep curriculum current. 

12. Have plan of continuing and systematic 
jattrnrl evaluation for improvement and development 
of program. 

13. Endorse and implement active leadership 
development program through related vocational youth 
organizations. 

14. Develop proper attitudes tov/ard work, good 
employer-employee relationships, and efficient v/ork 
habits and create the desire to continue personal and 
occupational development. 

15. Stress each student^s total development within 
his mental and physical abilities. 

16. Provide for satisfactory completion of defined 
program for at least 90 percent of students who do not 
transfer or withdraw for health reasons. 

17. Place students at the completion of their 
programs in the occupational "cluster" for which they 
are trained. 

18. Make an organized school placement service 
available to graduates. 

19. Keep public informed - have 'fJcirticipatory" 
support of community. 

20. Use any available means to acquaint students 
at all levels with vocational -technical career 
opportunities. 
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If you feel that any necessary components have been overlooked, please add and 
ra"ce them. 
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School ^ ^ . Program Area 



Student Teacher Principal Guidance Counselor 



SELF EVALUATION FOR VOCATIONAL-TECHNICAL PROGRAMS 



Please rate frankly your vocational-technical program using the following scale: 

5 Excellent; very well done 

4 Satisfactory; adequate 

3 Some improvements needed; no crucial weakness (es) 

2 Major improvements needed 

1 No effort made 

0 No opportunity to observe; not applicable 



1. The program objectives and requirements are known and 
supported by the faculty and other students. 

2. Vocational-technical students have an opportunity to 
participate in extra-curricular activities. 

3. Projects in yov: program are related to other subject 
area classes. 

4. Teachers and guidance counselors refer interested and 
qualified students to vocational-technical programs 
regardless of grade point averages. 

5. Interaction among administrators, guidance counselors, 
vocational teachers, and other teachers creates a good 
learning environment. 

6. Resources (time/ space/money) are sufficient for an 
effective vocational-technical program to be conducted. 

7. Vocational counseling services help students see their 
interests and aptitudes (as well as physical limitations) 
in light of occupational choices available. 

8. Vocational teachers, guidance counselors, and admin- 
istrators systematically examine the testing program 
for vocational students and make revisions where 
appropriate. 

9. Teachers and students review and understand test 
results as occupational plans are discussed. 
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Highj ! 

i i 

5 1 4 ! 3 

i ; 


Lov 
2 1 


r 

; 0 


10. Home visits are made to every interested vocational- 
technical student prior to enrollment. 




i 


* 1 

! i 

; 1 


11. Discussing prop.mm objectives enables the prospective 
vocational stuc'cnt and his parents to determine v/hether 
the objectives arc in lino with the student 'r, 
ability and occupational objectives. 

12. Parent- rtudent-teacher, parent-teacher, and/or student- 
teacher conferences are summarized and filed (treated 
confidentially) for future use. 


i 

j 

\ i 
I 

1 

1 1 
t \ 


i 


' } 1 

: 1 

; i 1 

j ' ( 
» 1 
I 

♦ • i 

1 ' i 


13. Student occupational objectives as defined in his 
program are periodically evaluated to determine if 
adjustments are needed. 


1 

1 

i 

1 
1 


1 

i 


] 1 

\ \ 

i 


14. Changes in a student *s program (after enrolled) can be 
made when consistent with his performance and after 

a conference with the teacher. i 

» . — 




1 


! 
i 




15. Administrators and guidance counselors assist the student { 
in making program changes when he indicates such a desire, i 












16. Tools, supplies, machines, and equipment are of suf- 
ficient number to conduct a quality program. 




; 


._ 


1 




17. Instructional materials used m addition to the test- j 
books are well selected, current, and easily acces- j 
sible. 1 










J 


18. Each student is provided adequate working space and * 
storage facilities. j 












19. The vocational teacher interprets the program to the i 
school and the community and assists in creating j 
good community relations. ; 












20. The teacher *s skill and content preparation for courses \ \ 
ne teaches are of the type and quality supported by the j 
"representative" Advisory Committee. i \ 

I 


! 
1 

_ J 


t 


1 

i 






1 I » 

21. The teacher belongs to and participates in local ; ! 
civic or other similar organizations. j 1 




t 

1 


1 




22. Occupational experience programs (cooperative, directed, ; ' 
or simulated) are provided for each student in relation j 
to his occupational objectives. j 

. .1., — « 






I 

.... i 


1 
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23. 



24. 



25. 



26. 



27. 



28. 



29. 



30. 



Written training plans are developed for each student's 
occupational experience program in relation to his 
occupational obiectives. 

Records are kept and summarized o*. the teacher's 
regular visits to training stations or on evaluations 
of simulated occupational experiences. 

An active Advisory Committeo representing fields in 
which vocational -technical students are receiving 
training has been organized. 

The Advisory Committee provides the teacher (s), admin- 
istrators, and the School Board information on current 
trends and developments in the community as they relate 
to vocational -technical programs. 

The "representative'' Advisory Committee assists in 
providing current occupational information, helping 
place students and graduates, establishing standards 
and evaluating the program. 

An annual follow-up survey of the previous year's 
graduates is made for the purpose of determining status 
regarding employment. 

Data collected from follow-up studies furnish evidence 
of how well vocational-technical educational objectives 
have been met and provide a basis for maintaining 
and improving the quality of vocational-technical 
services. 

Students are encouraged to reply to follow-up requests 
when contacted after graduation. 



High 
5 4 



Low 

3 2 I 1 . 0 




31. A local occupr mal survey is completed annually to 
determine empl. .ent opportunities available. 

32. Curriculum content is current and in line with the latest 
equipment and practices in the field of employment. 

33. The skills, knowledge, and attitudes required for 
employment in specific jobs are found in classwork, 
lab, experience program, and youth activities. 

34. There is an on-going evaluation program of objectives, 
content, methods, outcomes, and student performance. 



..A 




High 



35. Students, parents, teachers, business community leaders, 
and administrators are involved in the annual program 
evaluation. 

36. Findings from program evaluation ("feedback'^ bring about 
changes in the curriculum. 

37. Vocational youth organization activities are an integral 
part of the instructional program and the occupational 
experience program. 

38. Membership and participation in related vocational 
youth organizations are available to all^ students 
enrolled in the program. 

39. The activities of vocational youth organizations are 
planned, implemented, and evaluated by students. 

40. Each student is evaluated regularly on his work attitudes, 
relationship with the employer, work habits, and occupa- 
tional development. 

41. Students exhibit genuing pride in quality workmanship. 

42. Instruction is provided on human relations. 

43. Evaluation is made in terms of the individual's progress 
toward his own performance objectives not on the 
basis of comparing his performance with that of other 
students . 

44. The vocational student's self-image and total school 
achievement are favorably affected by the vocational 
educational experiences. 

45. Ethical practices and standards are goals of the program. 

46. A program of instruction is tailored to the needs and 
abilities of the individual students. 

47. At least 90 percent of students who do not transfer 
or withdraw for health reasons satisfactorily complete 
their defined programs. 

48. Factors such as absenteeism, tardiness, behavioral 
problans, and cost of supplies prevent students 
from completing their defined programs. 



, a 

J 
I 



■4 



.4... 



J 



^ 



I 
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49. An up-to-date file on job opportunities is maintained 
and made available to students in yoih* department. 

50. The teacher actively assists graduates in securing 
employment in the occupational area for which they are 

51. A high percentage of graduates are placed in the 
occupation for which they are trained. 

52. An organized school placement service is available to 
all vocational graduates. 

53. Vocational teachers actively seek the employment 
needs of the community. 

54. Students use sources other than the teacher and/or 
school to find entry-level jobs. 

55. Your program has a favorable reputation in the 
community and among employers. 

56. The "average" citizen knows about your program and 
what it has to offer. 

57. Employers support the program by recommending it to 
other students and other employers, donating equipment, 
furnishing consultants, sharing information and 
materials, ^tc. 

58. Vigorous efforts are made to insure that all students 
are informed of vocational -technical educatTonal 
opportunities and program requirements. 

59. Career occupation information has been developed at 
the pre- vocational grade level. 

60. Resource persons from all occupational areas are 
invited to assist in acquainting students with career 
opportunities. 



COMMENTS AND/OR QUESTIONS 
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EIGHTEEN SCHOOLS - PHASE II 



Central High School 
Bolivar, Tennessee 38008 

Bradley Central High School 
Occee Street 

Cleveland, Tennessee 37311 

Carter High School 
Route #2 

Strawberry Plains, Tennessee 37871 

Clinton Sr. High School 
Clinton, Tennessee 37716 

Doyle High School 

Route #10, Tipton Station Road 

Knoxville, Tennessee 37920 

Franklin County High School 
V/inchester, Tennessee 37398 

Green Brier High School 
Green Brier, Tennessee 37073 

Halls High School 
Emory Road 

Knoxville, Tennessee 37918 

Kirkman Technical High School 
?15 Chestnut Street 
Chattanooga, Tennessee 37402 



Lebanon High "chool 
Lebanon, Tennessee 37087 

McMinnville Area School 
Highway 70 South 
Mcffinnville, Tennessee 37110 

Mt. Pleasant High School 

Mt. Pleasant, Tennessee 38474 

Northside High School 
1212 Vollentine 
Memphis, Tennessee 38107 

Portland Senior High School 
Portland, Tennessee 

Rogersville High School 
Rogersville, Tennessee 37857 

South Fulton High School 
South Fultcn, Tennessee 42041 

Treadwell High School 
P20 N. Highland 
Memphis, Tennessee 38122 

Trousdale County Higii School 
Hartsville, Tennessee 37074 
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Teacher 

Program 

No. students enrolled: Seniors Juniors 



Other vocational- technical progi^ams in the school 



1. Are the program objectives written? (Get a copy if possible.) 

When were they revised? 

Are there levels for all types of students? 

2. Do vocational- /*chnical teachers confer and plan with general 
education teachers? 



3. Do you feel that you are treated fairly on salary (9/10/12 months* 
employment)? 

4. Do you feel that the administration and guidance counselors genuinely 
pror te your prograra? _; 

5. What tests are given to vocational students? How often? 

Interest Inventory 

Achievement ; 

Others 

6. How are results interpreted to teachers? 

To students? 

To parents? 

7. Are parents encouraged to visit classrooms? 



8. Do you have parent-teacher-student conferences before the student's 
program? During program? 

9* Do all vocational teachers cooperate in ifhe proper program placement 
of individual students? How? 



10. Can a student make a change within his program (after enrolled)? _ 

How is this accomplished? 

11. Is work space adequate? Are storage facilities provided? 

12. What teacher-prepared materials do^ou use? 



13. Do you hold a permanent certificate? 

area of instruction? 

When was your last paid occupational experience? 
^ Length? 

ERIC 



14. 



To which prof esslo'ial organizations do you belong? 



15. Do you use cooperative or directed work experience? 

Do you have simulated, occupational experience (lab)? 

Hox-7 often do yon visit work stations? 

Do you do any supervising at work stations? 

How many hours (on the average) do students work outside o£ 
school hours? 



16. Do you have an Advisory Committee? If not, why? 



How often do they meet? 

VHiat do they do? 

Provide information and speakers 
I'elp place st'itdents & graduates 
Help establish standards 
Assist in program evaluation 
Others 



17. Do you use follow-up surveys? 

Who handles these? 

What percentage of replies do you receive? 

Do yoii include dropouts? 

Are results used to suggest curriculum changes? 

18. How do you survey local employment opportunities? _ 



How often? 

How are findings related to curriculum offerings? 



19. Is your program internally evaluated? 

How often? 

Method (s) 



20. Do you have a vocational youth organization? If not, why? 



Who sponsors? 

Main problems encountered: 



21. Do you have joint conference(s) - student, cc^rdinator, employer? 
How often? 



22. How do you secure and use evaluations of the occupational experience 
part of the program? 



23. Have you been successful in outlining program objectives and ac- 
complishing them with disadvantaged and/or handicapped students 

In classes with "normal" students? 

Froblems? 



24. Do you observe any favorable changes in students' overall attitudes 
toward school, etc. after enrolling in the vocational-technical 
program? 

25. What percentage of your students complete defined programs? 

Why do others not complete their programs? 



26. Do you and/or the school feel responsible for placing graduates? 



27. How many graduates are employed where they got occupational 
experience? 

28. How many graduates take jobs in the "cluster" for which they trained? 



29. Do you have a placement service? 

If so, who coordinates it? 

30. How do you attempt to keep public informed about your program? 



33 . Does the community support your program? 
< If so, how? 



32. How do you and your school system acquaint students K-12 with career 
opportunities? 



Where do you currently begin? 

33. What percentage of applicants do you enroll in your program? 
COMMENTS 



Please list six (6) members of your Advisory Committee and/or 
employers, with their addresses and phone numbers. If you use 
neither in your program, list "lay" persons who would know about 
your program. 



1) Name 



Firm or Street 



City • Phone 



2) Name 



Firm or Street 



City Phone 



3) Name ^ 

Firm or Street 

City ^ Phone 



4) Name 

Firm or Street _ 

City Phone 



5) Name 



Firm or Street 



City Phone 



6) Name 



Firm or Street 



City 



Phone 
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